Resampling Imbalanced Class and the Effectiveness of Feature Selection Methods for Heart Failure Dataset

Clinical datasets commonly have an imbalanced class distribution and high dimensional variables. Imbalanced class means that one class is represented by a large number (majority) of samples more than another (minority) one in binary classification [1]. For example, in our research dataset there are 1459 instances classified as “Alive” while 485 are classified as “Dead”. Machine learning is generally predisposed by imbalanced data because most standard algorithms expect balanced class distributions, thereupon learning classification techniques achieve poorly for imbalanced data [1,2]. Many real world applications are critical for imbalanced data learning such as medical diagnosis, pattern recognition, and fraud detection [3]. Methods that can used to solve imbalanced data are categorized as the pre-processing approach and the algorithmic approach. The handling obtained by resampling the class distribution is by under-sampling the majority class, or over-sampling the minority class in the training set [2,4]. While boosting is an example of an algorithmic approach that recalculates weights with each iteration to place different weights on the training examples [5]. High dimensionality is one of the obstacles facing the mining of clinical data because high dimensionality causes high computational costs, difficulties interpreting data and may influence the classification performance. The dimensionality reduction categories have two types; feature extraction and feature selection. Feature extraction transforms the existing features into a lower dimensional space, for example, principal component analysis (PCA) and linear Discriminant analysis (LDA). Feature selection plays a crucial role in machine learning and pattern recognition [6]. It is generally the main data processing step prior to applying a learning algorithm [7]. Feature selection leads to reducing computation requirements, reducing the effect of the curse of dimensionality and developing the predictor performance [8].